Papers with cost-effective approach
Using Contextually Aligned Online Reviews to Measure LLMs’ Performance Disparities Across Language Varieties (2025.naacl-short)
Copied to clipboard
| Challenge: | Of the world's 7,000 languages, sixty (60) million people speak British English, 23 million speak Taiwan Mandarin, and 10 million speak European Portuguese. |
| Approach: | They propose a contextually aligned dataset that captures comments in different languages from real-world scenarios. |
| Outcome: | The proposed approach shows that large language models underperform in Taiwan Mandarin in a sentiment analysis task. |
MetaGPT: Merging Large Language Models Using Model Exclusive Task Arithmetic (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods face the trilemma of performance, data privacy, and computational costs, which hinders their application to LLMs. |
| Approach: | They propose a model-exclusive task arithmetic method for merging GPT-scale models which is data-agnostic and bypasses the heavy search process. |
| Outcome: | The proposed method achieves state-of-the-art performance on multiple tasks while minimizing the average loss difference between the merged model and each individual task model. |
Adversarial Math Word Problem Generation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) have revolutionized the educational landscape due to the great improvements in their natural language generation and problem-solving capabilities. |
| Approach: | They propose a cost-effective approach to attack large language models using abstract syntax trees to generate adversarial examples that preserve the structure and difficulty of the original questions aimed for assessment. |
| Outcome: | The proposed approach significantly degrades students' math problem-solving ability on open- and closed-source LLMs. |
Hard Negatives, Hard Lessons: Revisiting Training Data Quality for Robust Information Retrieval with LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Using LLMs to identify false negatives improves retrieval and reranker models by 0.7-1.4 points on BEIR and by 1.7-1.8 points on AIR-Bench evaluation. |
| Approach: | They use a simple, cost-effective approach to identify and relabel false negatives in training datasets. |
| Outcome: | The proposed approach improves retrieval models by 0.7-1.4 points on BEIR and by 1.7-1.8 points on AIR-Bench evaluation. |